Papers with systematic problems
Language Models as Inductive Reasoners (2024.eacl-long)
Copied to clipboard
| Challenge: | Inductive reasoning is a core component of human intelligence. |
| Approach: | They propose a task to induce natural language rules from natural language facts using natural language as representation for knowledge instead of formal language. |
| Outcome: | The proposed task surpasses baselines in both automatic and human evaluations. |
Automatically Exposing Problems with Neural Dialog Models (2021.emnlp-main)
Copied to clipboard
| Challenge: | Recent work suggests crowdworkers goad dialog models into generating unsafe and inconsistent responses, but humans leverage superficial clues such as hate speech, while leaving systematic problems undercover. |
| Approach: | They propose two methods to automatically trigger a dialog model into generating problematic responses by reinforcement learning. |
| Outcome: | The proposed methods expose safety and contradiction issues with state-of-the-art dialog models. |
Fixing Model Bugs with Natural Language Patches (2022.emnlp-main)
Copied to clipboard
| Challenge: | a growing body of research focused on using language to give instructions, supervision and even inductive biases to models instead of relying exclusively on labeled examples. |
| Approach: | They explore natural language patches that provide corrective feedback at the right level of abstraction. |
| Outcome: | The proposed model improves accuracy on real data by 1–4 accuracy points on different slices of a sentiment analysis dataset and F1 by 7 points on a relation extraction dataset. |